This paper focuses on the optimization problem min f(x) h(x), where f is of a finite sum structure (with n functions in the sum), with nonconvex but smooth components, and h is a convex but possibly nonsmooth function. So, this is a nonconvex finite sum problem with a convex regularizer. Function h is treated using a prox step. The authors propose a small modification to ProxSVRG (called ProxSVRG), and prove that this small modification has surprisingly interesting consequences. The modification consists in replacing the full gradient computation in the outer loop of ProxSVRG by an approximation thereof through subsampling/minibatch (batch size B).

Semantic Parsing aims to capture the meaning of a sentence and convert it into a logical, structured form. Previous studies show that semantic parsing enhances the performance of smaller models (e.g., BERT) on downstream tasks. However, it remains unclear whether the improvements extend similarly to LLMs. In this paper, our empirical findings reveal that, unlike smaller models, directly adding semantic parsing results into LLMs reduces their performance. To overcome this, we propose SENSE, a novel prompting approach that embeds semantic hints within the prompt. Experiments show that SENSE consistently improves LLMs' performance across various tasks, highlighting the potential of integrating semantic information to improve LLM capabilities.

Do Large Language Models (LLMs) make human-like linguistic generalizations? Dentella et al. (5) (DGL) prompt several LLMs ("Is the following sentence grammatically correct in English?") to elicit grammaticality judgments of 80 English sentences, concluding that LLMs demonstrate a "yes-response bias" and a "failure to distinguish grammatical from ungrammatical sentences". We re-evaluate LLM performance using well-established practices and find that DGL's data in fact provide evidence for just how well LLMs capture human linguistic judgments. Children learn to produce well-formed sentences without necessarily being able to articulate the underlying grammatical rules, a distinction long noted in linguistics (e.g., 1; 6; 3). DGL blur this distinction: their task requires not just grammatical competence, but also knowing what "grammatically correct" means.

Language models have become the backbone of today's AI systems. However, their predominant left-to-right generation limits the use of bidirectional context, which is essential for tasks that involve filling text in the middle. We propose the Fill-in Language Model (FiLM), a new language modeling approach that allows for flexible generation at any position without adhering to a specific generation order. Its training extends the masked language modeling objective by adopting varying mask probabilities sampled from the Beta distribution to enhance the generative capabilities of FiLM. During inference, FiLM can seamlessly insert missing phrases, sentences, or paragraphs, ensuring that the outputs are fluent and are coherent with the surrounding context. In both automatic and human evaluations, FiLM outperforms existing infilling methods that rely on left-to-right language models trained on rearranged text segments. FiLM is easy to implement and can be either trained from scratch or fine-tuned from a left-to-right language model. Notably, as the model size grows, FiLM's perplexity approaches that of strong left-to-right language models of similar sizes, indicating FiLM's scalability and potential as a large language model.

Advances in computational methods and big data availability have recently translated into breakthroughs in AI applications. With successes in bottom-up challenges partially overshadowing shortcomings, the 'human-like' performance of Large Language Models has raised the question of how linguistic performance is achieved by algorithms. Given systematic shortcomings in generalization across many AI systems, in this work we ask whether linguistic performance is indeed guided by language knowledge in Large Language Models. To this end, we prompt GPT-3 with a grammaticality judgement task and comprehension questions on less frequent constructions that are thus unlikely to form part of Large Language Models' training data. These included grammatical 'illusions', semantic anomalies, complex nested hierarchies and self-embeddings. GPT-3 failed for every prompt but one, often offering answers that show a critical lack of understanding even of high-frequency words used in these less frequent grammatical constructions. The present work sheds light on the boundaries of the alleged AI human-like linguistic competence and argues that, far from human-like, the next-word prediction abilities of LLMs may face issues of robustness, when pushed beyond training data.

Paid performance marketing is a true art, and one has to be experienced enough to make the right assumptions from the start. Usually, no one wants to spend too much money on experimenting, without an acceptable return on this investment in the end. If the first assumptions were wrong or keeping low-performance rates, a human marketer has to be able to optimize the ad at a very quick pace. There are already many places where AI can be applied in online marketing, and specifically in the paid performance marketing (PPC) field. The life cycle of a paid ad, or the "ad journey", i.e an ad that was created with the aim to be published on Google, Facebook, Instagram, LinkedIn, Twitter, Pinterest or any other platforms that provide paid targeted advertisement as a service, can be described as the following: Ad creation -- where the ideation process happens in a human's head (business owner or a marketer).

This September, financial institutions, venture capitalists, well established businesses, and startups alike joined together in New York City for 4 days of demos, panels, keynotes and roundtable discussions navigating the financial technology landscape. Of the 70 companies that demoed, AI dominated the discussion; with over 15% of companies insisting that "AI" is the driving force behind their tech. As the demos continued, it became increasingly clear that the disparities between those AIs are immense, and the extent to which their functionalities vary should not be overlooked. While Finovate 2016 outlined banks' need to implement AI into their platforms, Finovate 2017 identified several key factors that matter most when choosing a virtual banking assistant. In financial services, an industry overwhelmingly saturated with competition, customer service is the one true differentiator.